Papers with math reasoning tasks
LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can process extremely long contexts, requiring efficient inference over extended inputs. |
| Approach: | They propose a model that uses a constant-sized key-value cache to train long-context models. |
| Outcome: | Experimental results show that LongSpec achieves 3.26x speedup over strong Flash Attention baselines and 2.34x wall clock time on four math reasoning tasks. |
Plan, Verify and Switch: Integrated Reasoning with Diverse X-of-Thoughts (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for large language models (LLMs) have been used to prompt different reasoning thoughts, such as Chain of Thought and Program of Though. |
| Approach: | They propose a framework that prompts large language models with diverse reasoning thoughts by iterating between different prompting methods. |
| Outcome: | The proposed framework is able to generate multiple reasoning thoughts in 10 popular math reasoning datasets and is orthogonal to recent work that makes improvements on single reasoning methods and can generalise to logical reasoning domain. |
Self-Consistency Boosts Calibration for Math Reasoning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing solutions for math reasoning tasks use semantic parsing or AST decoding, but performance can degrade dramatically even with slight changes to the questions. |
| Approach: | They propose three calibration methods based on self-consistency for math reasoning tasks. |
| Outcome: | The proposed methods bridge model confidence and accuracy better than existing methods based on p(True) or logit. |
MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Low-rank adaptation and its mixture-of-experts (MOE) methods are highly effective but introduce significant latency in multi-tenant settings due to the LoRA modules and MOE routers added to multiple linear modules. |
| Approach: | They propose a low-rank adaptation variant that considers each LoRA module as an expert and employs a prompt-aware routing mechanism. |
| Outcome: | Extensive analysis on commonsense reasoning tasks and math reasoning tasks show that MiLoRA outperforms strong PEFT baselines with comparable tunable parameter budgets. |
Can LLMs Help Uncover Insights about LLMs? A Large-Scale, Evolving Literature Analysis of Frontier LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Recent surveys of literature highlight the overwhelming growth of Large Language Models (LLMs). |
| Approach: | They propose a semi-automated literature analysis approach that automates literature analysis using LLMs. |
| Outcome: | The proposed approach reduces paper surveying and data extraction by 93% compared to manual methods. |
IRIS: Interleaved Reinforcement with Incremental Staged Curriculum for Cross-Lingual Mathematical Reasoning (2026.acl-long)
Copied to clipboard
Navya Gupta, Rishitej Reddy Vyalla, Avinash Anand, Chhavi Kirtani, Erik Cambria, Zhengchen Zhang, Zhengkui Wang, Timothy Liu, Aik Beng Ng, Simon See, Rajiv Ratn Shah
| Challenge: | Curriculum learning fails to generate consistent step-by-step reasoning in multilingual and low-resource settings. |
| Approach: | They propose a framework that combines supervised fine-tuning with reverse curriculum reinforcement learning to generate consistent step-by-step reasoning. |
| Outcome: | The proposed framework outperforms single-axis benchmarks and multilingual test sets on math reasoning tasks and in high-resource languages. |
SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent large reasoning models have shown exceptional performance on various tasks, but they consume excessive tokens even for simple queries, leading to resource waste and prolonged user latency. |
| Approach: | They propose a self-adaptive reasoning strategy that automatically allocates budgets according to problem complexity and introduces GRPO for reinforcement learning to reduce output length. |
| Outcome: | The proposed model achieves an average response length compression of 61% on math reasoning tasks while maintaining accuracy. |
ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweighting (2025.acl-long)
Copied to clipboard
Rui Pan, Dylan Zhang, Hanning Zhang, Xingyuan Pan, Minrui Xu, Jipeng Zhang, Renjie Pi, Xiaoyu Wang, Tong Zhang
| Challenge: | Existing paradigms for bilevel optimization require second-order information, making it difficult to scale them up. |
| Approach: | They propose a scalable instantiation of a bilevel optimization paradigm for large-scale LLMs by using a memory-efficient training technique. |
| Outcome: | The proposed paradigm scales to 30B-sized LLMs on 8H100 GPUs. |
No Need for Explanations: LLMs can implicitly learn from mistakes in-context (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature assumes that correct answers to large language models must be accompanied by comprehensive rationales to be helpful. |
| Approach: | They propose to show incorrect answers to Large Language Models (LLMs) as a popular strategy to improve their performance in reasoning-intensive tasks. |
| Outcome: | The proposed approach outperforms chain-of-thought prompting in math reasoning tasks. |